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Introduction 



During the past four years, the data collection and reporting requirements of No Child Left 
Behind (NCLB) have stimulated the development or adoption of new education data systems by 
many states and districts. This work has been necessary, but it is not sufficient for data-driven 
decision making to make a mark on education at the local level. Data-driven educational decision 
making is more than a data system. It is a set of expectations and practices around the ongoing 
examination of student data to ascertain the effectiveness of educational activities and 
subsequently to refine programs and practices to improve outcomes for students. In this rapidly 
changing field, little is known about the prevalence of data-driven decision-making activities 
nationally or about the supports and barriers for putting these practices into place. 



Purpose of the Brief 

Using data from national surveys of teachers 
and school districts, this brief provides the first 
national estimates of the prevalence of K-12 
teachers’ access to and use of electronic student 
data management systems. Specifically, the 
brief addresses three research questions: 

• How broadly are student data systems being 
implemented in districts and schools? 

• Within these systems, how prevalent are 
tools for generating and acting on data? 

• How are school staff using student data 
systems? 



Data Sources 

This brief reports on analyses of two data sets from the U.S. Department of Education’s National 
Educational Technology Trends Study (NETTS). The primary data set used in this brief consists 
of responses of a nationally representative sample of K-12 teachers to a survey administered in 
fall and winter 2005. The teachers were clustered in schools sampled from districts participating 
in a NETTS district survey. The secondary data set consists of respondents to the NETTS district 
survey — a nationally representative sample of districts surveyed in spring 2005. Both district and 
teacher respondents were asked to report on activities during the 2004-05 school year. 

Teachers were sampled from 975 schools within districts selected for the NETTS district survey. 
High-poverty schools were oversampled to obtain more precise data about their technology use. 
The final teacher sample consisted of 6,017 teachers; 82 percent of them responded to the 
survey. The teacher survey collected information on teacher background characteristics; general 



What is data-driven 
educational decision making? 

In an education context, data-driven 
decision making is the analysis and 
use of student data and information 
concerning educational resources 
and processes to inform planning, 
resource allocation, student 
placement, and curriculum and 
instruction. The practice entails 
regular data collection and ongoing 
implementation of a continuous 
improvement process. 
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technology access, supports, and barriers; technology-related professional development; 
technology use by teachers and students; and items on the topic of this evaluation brief — the use 
of technology-supported student data management systems. Sampling weights were applied to 
obtain nationally representative estimates based on teacher responses. 

Districts were sampled from among the 12,483 districts that received federal Enhancing 
Education Through Technology (EETT) funds in 2003, as well as an additional 2,239 districts 
that had not received EETT funds. 1 The district survey was mailed to 1,039 district information 
technology professionals; 99 percent of them responded to the survey. The survey covered a 
variety of topics regarding the federal EETT program, including the availability of a technology- 
supported student data management system and its accessibility to teachers. 

Findings 

Access to a Student Data System 

Roughly half of all teachers (48 percent) reported having access to an electronic data system 
that provides them with student data. 

The first requisite for using an electronic student data system to support data-driven decision 
making is data access. Only about half of all K-12 teachers reported having access to an 
electronic student data system during the 2004-05 school year (Exhibit 1). 

Teachers in schools that were above average in the proportion of students eligible for free or 
reduced-price lunches were just as likely as those in lower-poverty schools to report having 
access to such a data system. 2 A somewhat lower proportion of elementary school teachers (45 
percent, compared with 54 percent of both middle and secondary school teachers) reported 
having access to a student data system. 



1 The 60 largest urban districts were selected with certainty; districts composed entirely of special education schools 
and vocational-technical schools and independent charter schools that are their own districts, were excluded from 
the district sampling frame. To obtain the NETTS teacher survey sample, schools were selected from the districts 
in the district survey, and then a stratified sample of teachers was selected from the sampled schools (schools were 
stratified by poverty and grade level). Although the sampling process was sequential, entities at each level were 
selected at random (e.g., teachers were randomly selected from staff rosters from each of the schools in the 
sample). 

2 For schools, “higher poverty” was defined as above a specified cutoff in terms of the percentage of students who 
were eligible for the free or reduced-price lunch program. The dividing line between higher-poverty and lower- 
poverty schools was selected to ensure that for each school type (elementary, middle, or high school), there would 
be the same number of teachers in the higher-poverty and the lower-poverty groups, as reported in the NCES 
Common Core of Data (CCD). Elementary schools with 29.7 percent of their students eligible for free or reduced- 
price lunches were classified as higher poverty. For middle and high schools, the poverty thresholds were 24.3 
percent and 15.9 percent, respectively. 
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Of the teachers who said they had access to a 
student data system, the great majority (75 
percent) reported that it was provided by their 
district. Some teachers (15 percent) reported 
having access to a student data system made 
available by their school; relatively few 
teachers (4 percent) reported having a system 
provided by their state; 6 percent did not know 
the source of the system to which they had 
access. 



What is an electronic student 
data system? 

The collection of wide-scale 
information on the availability and 
use of electronic student data 
systems is hampered by the lack of a 
generally accepted definitions for a 
“student data system.” Generally 
speaking, an electronic data system 
is a collection of programs 
supporting the digital storage, 
manipulation, and extraction of 
information from a database. In 
addition to housing current and 
historical data on students, some data 
systems include programs 
capable of capturing transactions 
such as attendance, managing 
curriculum resources, and analyzing 
student data. 



Exhibit 1 

Percentage of Teachers Reporting Access to a Student Data System 
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Exhibit reads: Overall, 48 percent of teachers said that they had access to a 
student data system in school year 2004-05. 

Note: * indicates significant difference between teachers in elementary schools 
and those in middle or high schools. 

Source: NETTS teacher survey, 2005. 
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More than 60 percent of districts reported having made electronically stored student data 
available to their teachers in 2004-05. 

Among the districts responding to the NETTS survey of local education agencies, 62 percent 
said that they had electronic data that teachers could access. But many teachers in these districts 
were unaware that there was a system they could access. Among those districts in which teachers 
were sampled for the NETTS teacher survey, just 53 percent of teachers in districts that reported 
providing electronic data that teachers could use reported having access to an electronic student 
data system. 3 It is quite possible that teachers were unaware of the data access provided by their 
district or that teachers lacked the hardware, network connection, or know-how to access a 
system theoretically available to them. Another possibility is that many teachers regard a student 
data system as something more than access to district- stored student data. This problem could 
have been exacerbated by the fact that the NETTS district and teacher surveys used somewhat 
different terminology. 4 

There were also 393 teachers who said they had access to a district-provided student data 
management system, although their districts reported that they did not have student data stored 
electronically in a system that teachers could access. Because the extent to which this 
discrepancy reflects teacher misconceptions concerning the source of a system, differences in 
question interpretation, or misreporting by the district or the teacher is unknown, the teacher 
survey data were analyzed both with and without these 393 cases. The trends and patterns 
reported in this brief do not appear to be strongly influenced by responses of this set of teachers. 
When the analyses reported in this brief were rerun without these 393 cases, only a single 
statistic changed significantly (and that change was just 2 percentage points). 5 



As of 2004-05, over two-thirds of all school districts maintained electronic records of students’ 
standardized test scores. 

Based on all districts responding to the NETTS district survey, the most frequent types of 
electronic data maintained by districts were attendance (99 percent), demographics (94 percent), 
course enrollment histories (93 percent), special education information (92 percent), and grades 
(89 percent). (See Exhibit 2.) Standardized test scores were maintained in electronic form by 69 
percent of districts. Fewer districts stored electronic data on teacher qualifications (49 percent) 



3 This statistic is based on a subsample of 448 districts from which participating schools and teachers were drawn. 

4 The district survey asked whether the district stored specific types of data electronically and then whether teachers 
could access each type of electronically stored data. The teacher survey asked whether the teacher had access to 
“an electronic data management system that provided you with student data.” 

5 The numbers reported in this brief include responses of these 393 teachers; the alternate value for the statistic that 
would change significantly if these cases were excluded is provided in a table note for Exhibit 8. 
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and on participation of students in particular educational programs, such as those using an 
innovative classroom curriculum (37 percent). 



Exhibit 2 

Types of Data That Districts Store Electronically 




Exhibit reads: Almost all districts (99 percent) stored attendance data in 
electronic form; 69 percent stored student standardized test scores 
electronically. 

Source: NETTS district survey, 2005. 



Even though nearly half of all teachers (48 percent) reported having access to student data 
systems, they did not necessarily have the information or tools they needed to make use of the 
student data available to them. 

There is typically a lag between standardized test administration (usually done in the spring) and 
availability of student data. Researchers have noted teachers’ frustration with data systems that 
do not include achievement measures for the group of students with whom they are currently 
working (Thom, 2002). Less than 40 percent of teachers with access to a student data system 
reported having access to standardized test scores from the current year for their students 
(2004-05 data in Exhibit 3). Given that only 48 percent of teachers reported having access to a 
student data system, this means that just 19 percent of responding teachers said that they had 
access to current-year test scores for their students. Even fewer teachers (less than 20 percent of 
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the 48 percent with data access) reported having access to software that they could use to analyze 
and interpret test scores. Around a fifth of teachers (22 percent) with access to a student data 
system reported that it included course enrollment histories for their students. 

Only a small proportion of the teachers with access to a student data system reported that the 
system incorporated online assessments (17 percent) or online instruction that students could use 
(13 percent). Just 6 percent of teachers with access to a student data system said that it provided 
information on students’ participation in supplementary education programs such as tutoring. 

The only two types of data that a majority of teachers with access to a student data system 
reported being able to access were class attendance and course grades (74 percent and 66 
percent, respectively). 
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Exhibit 3 



Teacher Reported Categories of Data and Support 
Available to Them 




Percent of teachers with access to student data system 

Exhibit reads: Of the 48 percent of teachers who reported having access to a 
student data system, 74 percent indicated that they had access to attendance 
data, and 39 percent indicated access to the current year’s (2004-05) 
students’ standardized test scores. 

Note: <2003-04 indicates access to standardized test scores from years prior 
to 2003-04. 

Source: NETTS teacher survey, 2005. 
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How Teachers and Districts Use Student Data Systems 



Teachers are using student data systems to provide information to parents, to monitor their 
students’ status with respect to accountability requirements, and to plan and refine their 
instruction. 

A large majority (89 percent) of teachers with access to a student data system said that they used 
the system at least a few times during 2004-05. Given the fact that only 48 percent of teachers 
reported having access to a student data system in 2004-05, this means that just 43 percent of 
teachers were using a data system in that school year. When system use for particular purposes is 
examined, the most commonly reported use of data systems was to inform parents about their 
students’ progress. Seventy percent of the teachers who had acess to student data systems had 
used them for this purpose (Exhibit 4). Similarly, 65 percent of the teachers who had access to 
student data systems had used them to track individual students’ test scores. About 44 percent of 
the teachers with system access had used the systems to examine test scores for a class of 
students. 

Exhibit 4 

Percentage of Teachers Who Reported Using a Student 
Data System at Least a Few Times a Year for a Specific 

Function 



0% 20% 40% 60% 80% 100% 

Percent of teachers with access to student data system 




Inform parents 
Track individual test scores 
Track other measures 
Identify skill gaps 
Inform curriculum changes 

Pacing 

Track test scores by grade 
Student placement 
Identify promising practices 
Track AYP 
Assess test-taking needs 



Exhibit reads: Of the 48 percent of teachers who reported having 
access to a data system, 70 percent said that they had used system data 
to inform parents of student progress; 55 percent said that they had 
used system data to identify individual students’ skill gaps. 

Source: NETTS teacher survey, 2005. 






Use of student data systems to plan and individualize instruction appears less common than 
use of the systems to inform parents or keep track of accountability measures. 

Just over half (55 percent) of the teachers with access to student data systems reported having 
used system data to identify individual students’ skill gaps, 50 percent had used data from the 
systems in planning changes to the curriculum, and 41 percent had used data to evaluate 
promising classroom practices. The ways in which teachers use the data systems may be 
influenced by the types of data and data query functions in the systems available to teachers 
(Wayman et al., 2004). 

Tracking accountability measures is also the primary use that districts make of their student 
data systems. 

The NETTS district survey asked district staff to report the purposes for which they used their 
student data systems. Like teachers, district staff report using the system to track test scores and 
other measures of student progress, for student placement, and to inform parents of student 
progress (Exhibit 5). Similar proportions of district and teacher respondents reported using a 
student data system to help inform curriculum changes (61 percent and 50 percent, respectively). 
Forty-one percent of districts with student information systems used student data from their 
systems in evaluating promising practices, and 28 percent used data systems in evaluating 
teacher performance. 
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Exhibit 5 



Percent of Districts Using a Data System at Least a Few Times a 
Year for a Specific Function 




Exhibit reads: Among districts storing data electronically, 80 percent reported 
using the system to track other measures of student progress; 61 percent 
reported using data systems to inform curriculum changes. 

Note: Other student progress measures are measures other than test scores and 
graduation rates. 

Source: NETTS district survey. 2005. 
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There were few differences by grade level and subject area in how teachers reported using 
student data systems. 

Using five teacher grade or subject categories (elementary generalist, middle and high school 
English, middle and high school mathematics, middle and high school science, and middle and 
high school social studies), statistically significant differences were found in the prevalence of 
just three of the 1 1 data system uses examined in the survey. 6 As shown in Exhibit 6, differences 
were found in the following uses: 

• Identifying skill gaps. Teacher groups varied in their propensity to use data systems to 
identify their students’ skill gaps. The difference between elementary teachers (61 
percent), who were the group most likely to do so, and middle and high school 
mathematics teachers (43 percent), who were the group least likely to do so, was 
statistically significant. 

• Tracking other measures. Different teacher groups varied in their use of data systems to 
track measures of student progress other than standardized test scores. Reported use of 
this practice varied from a high of 68 percent for middle and high school history and 
social studies teachers with access to student data systems to a low of 46 percent for 
mathematics teachers. 

• Pacing. Among teachers with access to a data system, there were differences across 
teacher groups in the proportion using student data to help determine how to pace 
instruction. Elementary teachers (54 percent) appeared to be most likely to report using a 
data management system to help in pacing instruction; middle and high school 
mathematics teachers and history and social studies teachers appeared least and likely (40 
percent) to report doing so, but the difference did not attain statistical significance. 



6 For each system use, an omnibus test for differences in the proportion of teachers using the system in that way was 
conducted. For the three uses for which the omnibus test was significant at p<.05, every possible pair-wise 
combination of teacher categories was examined. Significance levels for the post-hoc tests were set using the 
Benjamin and Hochberg (1995) procedure for controlling false discovery rates. Only one of the individual 
pairwise comparisons (the difference between elementary and middle and high school math teachers using a 
student data management system to identify skill gaps) attained significance. 
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Exhibit 6 



Percentage of Teachers Who Reported Using a Student Data System at Least a Few 
Times a Year for a Specific Function, by Teaching Area 

Middle/High School 

History or 

Elementary Social 



System Use 


School 


English 


Math 


Science 


Studies 


Inform parents 


66 


74 


69 


68 


78 


Track individual test scores 


65 


62 


69 


56 


65 


Track other measures* 


63 


58 


46 


51 


68 


Identify skill gaps* 


61* 


47 


43* 


46 


49 


Inform curriculum 


51 


52 


42 


51 


55 


Pacing* 


54 


52 


40 


41 


40 


Track test scores by grade 


44 


43 


38 


44 


48 


Student placement 


40 


39 


54 


33 


46 


Identify promising practices 


42 


43 


33 


44 


43 


Track AYP 


39 


35 


39 


33 


30 


Assess test-taking needs 


35 


48 


28 


37 


40 



* Use varies significantly across the five teacher categories. 

Percentage using system to identify skill gaps differs significantly between elementary teachers and 
middle and high school math teachers. 



Exhibit reads: Among teachers who reported using a student data system, 66 percent of elementary 
teachers said that they used the system to inform parents about student progress. Among middle and 
high school teachers, 74 percent of English teachers, 69 percent of math teachers, 68 percent of 
science teachers, and 78 percent of history and social studies teachers said that they used the system 
to inform parents about student progress. 

Source: NETTS teacher survey, 2005. 



Teachers in higher-poverty schools reported using their student data system at least as 
frequently as did teachers in lower-poverty schools. 

The poverty level of the school tends to have some association with frequency of data system use 
for all the purposes examined (Exhibit 7), but the difference between teachers from higher- 
poverty and lower-poverty schools was statistically significant for only three of the 1 1 uses 
included in the survey. Teachers at higher-poverty schools were more likely than their colleagues 
at lower-poverty schools to use student data systems to track grade-level test scores, to identify 
promising teaching practices, and to assess their students’ need for test-taking practice. 



12 





Years of teaching experience appear to have little influence on the purposes for which teachers 
use student data systems; the only significant difference in data system use related to years of 
teaching was a greater likelihood that teachers with the most experience (9 or more years) would 
use the student data system to inform parents about student progress. 



Exhibit 7 



Teachers Who Reported Using a Data 
Management System at Least a Few Times a Year for a Specific 
Function, by Poverty Level of School 
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Exhibit reads: Of the teachers who reported access to a student data 
system, 42 percent of teachers in higher-poverty schools reported using 
the data system to assess students’ need for test-taking practice, while 
29 percent of teachers in lower-poverty schools did so. 

Note: * indicates significant difference between teachers in higher-poverty 
and those in lower-poverty schools. 

Source: NETTS teacher survey, 2005. 
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Support for Using Student Data Systems 

Teachers reported that using the student data system in collaboration with school colleagues 
was almost as common as using the system on their own. 

The social context in which teachers use data systems (whether as a solitary activity or in 
collaboration with colleagues) provides an indication of the kinds of decisions being addressed 
with data. Teachers who use student data to make instructional decisions for their own class or 
for individual students in their class are likely to spend time examining data on their own. 
Teachers who are engaged in making broader decisions, such as those related to a school 
improvement plan or department- or grade-level decisions about curriculum and instruction are 
likely to look at data with colleagues. The data-driven decision-making movement promotes both 
kinds of activity, and past research suggests that a school culture stressing collaboration 
facilitates the implementation of data-driven decision making (Chen, Heritage, and Lee, 2005). 
Around three-quarters (77 percent) of the teachers with access to a student data system said that 
they had used it on their own; 71 percent said that they had used it in a joint activity with other 
staff from their school (Exhibit 8). 

Nearly 60 percent of the teachers with access to a student data system reported having used the 
system as part of a district activity in their school and 27 percent as part of a district activity with 
staff from other schools. Less than 5 percent reported having used the system in a social context 
other than the four included in Exhibit 8. 
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Exhibit 8 



Teachers Who Reported Using a Data 
System at Least a Few Times a Year to Make 
Instructional Decisions, by Social Context of Use 




Exhibit reads: Of the 48 percent of teachers who reported access to a 
student data system, 77 percent said they had used the system on their 
own and 7 1 percent said they had used it with other teachers at their 
school; 58 percent said that they had used the system as part of a district 
activity conducted at their school. 

Note: If teachers who reported having access to a district-provided 
system in districts that said they did not provide teachers with access to 
student data systems are deleted from the analysis, the percentage of 
teachers saying they had analyzed data on their own is reduced to 75 
percent. 

Source: NETTS teacher survey, 2005. 
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The most common supports for teachers’ use of student data systems come from their schools 
and not through formal course work. 

Over 60 percent of teachers with access to student data systems reported having received 
professional development on this topic at their school; a similar proportion reported that their 
principals supported data-driven decision making (Exhibit 9). In contrast, less than 10 percent of 
teachers with access to systems reported having had formal course work on the use of student 
data systems. 

Roughly a quarter (28 percent) of teachers who had access to student data systems said they had 
received support from a consultant or mentor teacher for this activity. Only 8 percent of teachers 
reported having had formal course work on data-driven decision-making. One-sixth (16 percent) 
of teachers with access to a data system reported having had paid time set aside for examining 
student data and using data to make decisions about practice. (This would include time during 
the regular school day or regular professional development days covered by their salary, as well 
as any summer or after-school sessions for which they received a stipend.) 

Teachers in higher-poverty schools were as likely as those in lower-poverty schools to report 
each type of support for using student data systems. Mathematics teachers were less likely than 
other teachers to report having received professional development for use of student data 
systems, and teachers with nine or more years of teaching experience were also less likely than 
their less-experienced colleagues to report having had this kind of professional development. 
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Exhibit 9 



Teachers Indicating Support for Using 
Student Data to Guide Instruction, by Source of Support 




Percent of teachers with access to student data system 



Exhibit reads: Of the 48 percent of teachers who reported having access 
to a student data system, 63 percent had received professional 
development at school; 16 percent reported having had paid time set 
aside for student data use. 

Source: NETTS teacher survey, 2005. 



Summary 

The data reported in this brief provide a snapshot of teachers’ access to and supports for using 
student data systems in the 2004-05 school year. They provide a baseline against which 
outcomes associated with new federal, state, and district efforts to promote the use of data 
systems to improve instruction and student achievement can be compared. 

About half of all teachers reported having access to an electronic student data management 
system in school year 2004-05 (but there is some ambiguity concerning the meaning of 
“access” and “data management system,” as demonstrated by apparent discrepancies between 
teacher and district responses). Of those teachers who reported having access to a data system, 
a majority reported making use of it for monitoring student progress, planning, and refining 
their instruction, or keeping parents informed (all of which are promoted under NCLB). 
Teachers in higher-poverty schools were no less likely than their counterparts in lower- 
poverty schools to report having access to and using student data systems, and for some 
purposes they reported more frequent data use than did teachers at lower-poverty schools. The 
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survey data suggest also that teacher collaboration around student data system use is common. 
Teachers were nearly as likely to use their student information system with their colleagues as 
on their own. More than half of teachers with access to student data systems reported that they 
had received professional development and principal support to use the data system — two key 
support elements. 

At the same time, in the 2004-05 school year teachers did not necessarily have appropriate 
data or the tools they needed to make good use of student data in planning and individualizing 
instruction. Less than half of the teachers who reported having had access to a student data 
system said that the system had student test scores from the current year. Few teachers 
reported having software to help them analyze and interpret test scores, and few had received 
formal training or course work on the use of student data to inform instruction. Some teachers 
had received support from a consultant or mentor teacher. Differences in the use of student 
data systems by grade level and subject area imply different levels of need, related to varying 
uses of data and differences in teachers’ current skill sets. In 2004-05, the majority of 
teachers had yet to use data to plan and individualize instruction. Previous research examining 
data-driven decision making in individual schools has suggested that the lack of time 
allocated to collecting, analyzing, and interpreting data is a major barrier to effective 
implementation of this practice (Ingram, Louis, and Schroeder, 2004; Marsh, Pane, and 
Hamilton, 2006). On the NETTS survey, only about one-sixth of teachers with access to data 
systems reported that they had had paid time set aside for data use. Hence, for the most part, 
districts are in essence expecting teachers to use the data system as an extra activity, either 
conducted on their own time or substituted for other activities during their regular planning 
period. 

District staff will need to confront these issues as they review their data-driven decision- 
making practices and plan support and professional development activities, and by teacher 
preparation staff as they assess their teacher training programs. 
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